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A queue is required when a service provider is not able to handle jobs arriving over the time. In a highly 
flexible and dynamic environment, some jobs might demand for faster execution at run-time especially when 
the resources are limited and the jobs are competing for acquiring resources. A user might demand for speed 
up (reduced wait time) for some of the jobs present in the queue at run time. In such cases, it is required 
to accelerate (directly sending the job to the server) urgent jobs (requesting for speed up) ahead of other 
jobs present in the queue for an earlier completion of urgent jobs. Under the assumption of no additional 
resources, such acceleration of jobs would result in slowing down of other jobs present in the queue. In this 
paper, we formulate the problem of Speed Up Scheduling without acquiring any additional resources for the 
scheduling of on-line speed up requests posed by a user at run-time and present algorithms for the same. 

We apply the idea of Speed Up Scheduling to two different domains - Web Scheduling and CPU Schedul¬ 
ing. We demonstrate our results with a simulation based model using trace driven workload and synthetic 
datasets to show the usefulness of Speed Up scheduling. Speed Up provides a new way of addressing ur¬ 
gent jobs, provides a different evaluation criteria for comparing scheduling algorithms and has practical 
applications. 
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1. INTRODUCTION 

One does not like to wait. In most of the cases, the number of service providers are 
less compared to the number of jobs requiring service. In such cases, a queue is re¬ 
quired because the service provider is not able to handle jobs as soon as they arrive. 
There are many practical scenarios where queuing cannot be avoided. For example, 
accumulation of web requests in a queue arriving at web server, a queue of packets 
at routers, a queue of processes waiting for CPU resources etc. In such situations, the 
choice of scheduling policy determines mean wait time, mean queue length and other 
performance measures. 

In real life, job execution often requires human intervention in its execution. Fur- 


Author’s addresses: International Institute of Information Technology, Hyderabad 500032 India. 

Author’s email addresses: yash.guptaug08@students.iiit.ac.in (Yash Gupta) and kamal@iiit.ac.in (Ka¬ 
malakar Karlapalem). 

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted 
without fee provided that copies are not made or distributed for profit or commercial advantage and that 
copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights 
for components of this work owned by others than ACM must be honored. Abstracting with credit is per¬ 
mitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component 
of this work in other works requires prior specific permission and/or a fee. Permissions may be requested 
from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax -|-1 (212) 
869-0481, or permissions@acm.org. 

© YYYY ACM OOOO-OOOO/YYYY/Ol-ARTA $15.00 
DDI:http://dx.doi.org/10.1145/0000000.0000000 


ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY. 




A:2 


ther in a highly dynamic environment, a particular joh may raise the requirement for 
urgent attention, especially when the resources are limited. The user may demand for 
speed up (reduced wait time/faster execution) for some specific job present in the queue 
at run time. In such a case, we need to accelerate/speed up such job (that requested 
for speed up based on user demand) ahead of other jobs present in the queue to the 
server for service. However, such acceleration would result in slowing down of other 
jobs which are present ahead of it in the queue since no additional resources are used. 
Thus, the problem that arises is how to schedule such speed up requests posed by the 
user so as to facilitate their earlier execution but at the same time ensuring less im¬ 
pact of slow down on the other jobs. 

In modern connected world, there is increasing prevalence of cloud computing 
which can dynamically allocate resources owing to an application need. A job seeking 
urgent attention can be served by dynamically allocating more resources to it. How¬ 
ever, resources are not infinite. Even in such scenarios, there could be high possibility 
of tremendous resource contention. It is not always feasible to provide additional re¬ 
sources to an urgent job either due to unavailability or high cost of resources. In such 
cases, speed up quickly process all the urgent jobs without acquiring any additional 
resources. 

Consider an example of CPU Scheduling of different processes. The processes are 
usually categorized into different priority classes. All the processes of same priority 
class are usually scheduled using first come first serve or Round Robin strategy. In 
such a case, it is possible for a user to request speed up for a specific user process. 
However, the requested speed up is not urgent enough to move the process to higher 
priority class but the process needs to be accelerated with respect to those having same 
priority. Under the assumption of limited resources, some other processes within same 
priority class have to be delayed in order for one user to achieve requested speed up. 

The idea behind speed up is to achieve faster execution of jobs (requested by the 
user at run-time) without acquiring additional resources but with an overhead of some 
other jobs getting delayed. Yet we want to facilitate earlier execution of urgent jobs 
(whose delay might have severe consequences) without unnecessarily slowing down 
the other jobs. It is important to note that it might not be possible to achieve speed up 
for all the jobs that requested for it as well as it might be the case that while speeding 
up some jobs, even some of the jobs that requested for speed up were slowed down 
IKafeza et al. 200^ . 

1.1. Background and Related Work 

Regarding scheduling in real time systems there is lot of work done for priority, 
job deadlines, data deadlines, precedence constraints etc. but in every case once the 
ordering is assigned and a job is scheduled it cannot be altered. In our speed up 
problem execution needs to be adjusted at run time in order to respond to on-line user 
requests for speed up. The job priority is not determined according to a predefined 
scheduling policy assignment criterion like real time scheduling policies, but according 
to the user on-line requests for speed up posed at run-time. 

Various scheduling techniques have already been proposed such as First Come 
First Serve, Shortest Job First, Priority Scheduling, EDF (Earliest Deadline First), 
Multilevel Queue scheduling. Round Robin etc. These scheduling policies perform 
effectively in improving the performance of the system in the absence of on-line speed 
up requests, but when such requests occur we need more fine grained scheduling 
policies which can speed up all the activities that requested for it at run time. We 
can compare and contrast different scheduling algorithms based on speed up/slow 
down characteristics. The results and analysis for such comparisons are presented 
and explained in later sections of this paper. When a specific job completes before 
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its expected execution time with respect to a specific scheduler, then we refer this as 
speeding up of job. Shortest Job First speeds up the job with shorter duration time, 
Priority scheduling speeds up the job with highest priority both with respect to FCFS 
scheduler. However these scheduling policies are unable to handle on-line speed up 
requests where execution needs to be adjusted at run time to provide remedies for 
the delayed and urgent jobs. Further existing scheduling policies speed up specific 
kind of jobs which do not take into consideration the slow down for other jobs. In our 
problem of speed up, we might have to speed up even the delayed non-urgent jobs 
under certain circumstances. 

Earliest Deadline First (EDF) scheduling policy focuses on speeding up of jobs 
with earliest deadlines so that they do not miss their deadlines. EDF scheduling 
problem seems similar to speed up problem in a sense that it can be executed on-line, 
the priority depends on the jobs that arrive (specific deadlines) and the priority 
assignment is done at run-time but there is a difference between preserving deadlines 
and problem of speed up. The notion of deadline is absolute whereas concept of speed 
up is relative. The deadlines of arriving jobs are absolute and independent of the 
state of the queue at their arrival whereas the concept of speed up is defined with 
respect to expected execution time which is influenced by the queue state. In EDF, 
the deadlines are met by speeding up of urgent jobs but in our case, there need not 
be any deadline to achieve speed up. EDF tries to minimize the number of jobs that 
miss their deadlines, whereas Speed Up focus is to accelerate/speed up as many jobs 
as possible which requested for it at run-time. Further, EDF scheduling does not care 
about the jobs which do not have specific deadlines (non urgent), but in our case we 
cannot penalize more than necessary to the rest of the executing jobs. In speed up 
problem, it is more beneficial to achieve the exact requested speed up for urgent job 
rather than achieving relatively high amount of speed up (than requested) thereby 
causing higher amount of slow down to the rest of the executing jobs. 

Speed Up provides a new way of addressing urgent jobs, provides a 
different evaluation criteria for comparing scheduling algorithms and has 
practical applic ations. There has been litt le work on Speed Up schedul ¬ 
ing except IKafeza and Karlapa lem 2000b|, [Kafeza and Karlapalem 2000a) , 
Karlanalem 200111 and liKafeza et al. 200611. T' ^ ~ 


IKafeza and Karlanalem 200111 and liKafeza et al. 200611 . The authors modeled speed 
up problem for speeding up of E-commerce activities | Kafeza and Karlapalem 2000b) 
and improving the response time of business nrocesses IIKafeza et al. 200611 . The au- 
thors presented three explicit speed up algorithms (MPF, MPF-SD and MinPF) based 
on location table model where they select a specific job based on its queue position and 
achieve speed up by swapping of jobs in the queue. MPF (Maximum Position First) 
algorithm tries to push forward all the jobs requesting for speed up at the head of the 
queue and decides their order by trying to swap small positions (near to the head) 
with the large ones (end of the queue). MPF-SD (MPF-Smaller Duration) algorithm is 
similar to MPF algorithm but it preserves one property that “ no job that is requesting 
for speed up is delayed”. The urgent job (requesting for speed up) is pushed ahead 
in the queue if and only if its duration is smaller than the one with which it swaps. 
MinPF (Minimum Position First) algorithm is compliment of MPF algorithm. Instead 
of swapping jobs with maximum distance, it swaps jobs with minimum distance. All 
the three algorithms are purely positional and computation ally expensive wit h an 
overhead of maintaining a locat ion table. Further, authors in IIKafeza et al. 200^ and 
I Kafeza and Karlapalem 2000b | used only achieved ratio metric (percentage of jobs 
that achieved their requested speed up) for evaluating the effectiveness of their speed 
up algorithms. Their work did not consider the impact of slow down on the remaining 
jobs as a result of speed up which is crucial. Any scheduling algorithm speeding up 
some urgent jobs requiring urgent attention but causing arbitrarily high slow down to 
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the rest of the jobs would not be practical and fair. Therefore along with achieved ratio, 
we also consider the impact of slow down to the non urgent jobs as a consequence 
of speed up and the overall mean wait time. In this paper, we remodel the speed up 
problem aiming at speeding up the jobs which requested for it but at the same time 
providing less scope of slow down for the rest of the executing jobs while keeping 
the mean wait time reasonable. We provide implicit techniques to address speed up 
problem where the notion of acceleration is incorporated in the priority function, thus 
leading to computationally efficient solution. 

The rest of the paper is structured as follows. The Speed Up scheduling problem is 
formulated in Section 2 and Speed Up algorithms are presented in Section 3. Section 
4 presents the experimental analysis of our proposed algorithms. Section 5 and 6 
respectively describes about the applications of Speed Up in web scheduling and CPU 
scheduling. Section 7 concludes our work along with possible future work. 

2. SPEED UP PROBLEM 

Throughout this work, a single server queuing model is used for addressing the 
problem of speed up. Even in the presence of multiple queues, we need to support 
speed up within every single queue associated with a specific server. Load balancing 
(movement of jobs across different queues to equally distribute workload on multiple 
servers) is a separate problem which may or may not help in achieving speed up (when 
the queues are equally loaded). Supporting speed up through load balancers is beyond 
the scope of this work. 

Consider jobs arriving at a system with respect to time. The system comprises of a 
single server which can serve at most one job at any point of time and a queue where 
the jobs accumulate and wait for their service. The jobs are assumed to be non pre- 
emptable in this work. The job once finished leaves the system. The state of the system 
is a snapshot of the queue at a particular instant of time. 

Definition 2.1 {System State). State of the system at any point of time t denoted by 

S{t), is defined as the tuple < . ,ji > where I is the queue length at time t 

and ji is the job at ith position waiting in the queue for service, such that 1 <i < I and 
each job ji having arrival time strictly less than t. The job getting its service at t is not 
included in S{t). 

The concept of speed up by itself is relative IIKafeza et al. 20061 . Therefore, we de¬ 
fine an expected execution time for each job j and then define speed up with respect to 
it. The expected execution time for job j (expected total time spent in the system from 
its arrival till it is completely finished) with respect to scheduling policy P is defined 
as the summation of the processing time of all the jobs that would be executed before 
j depending upon P plus the processing time of job j. Note that the processing time of 
job is the service time of job also referred to as duration of job. If P scheduling policy is 
assumed to be FCFS, then expected execution time for job j would be defined as sum¬ 
mation of processing time of all the jobs present in the queue which arrived before j 
plus the processing time of j. Note that, the definition of expected execution time would 
change if some other scheduling technique is assumed. Throughout this work, we use 
the term expected execution time with respect to FCFS scheduler. 

Definition 2.2 {Expected Execution Time). The expected execution time for job j 
with respect to FCFS scheduler arriving at time t, denoted by Texp{j) is defined as 



( 1 ) 


Vfees(t) 
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where d{k) denotes the processing time/duration for joh k and drem{js) denotes the 
remaining processing time of job js being served at the server. If there is no job at 
server being served at t, then value of dj.emi.3s) is zero. 

Definition 2.3 {Actual Execution Time). The actual execution time for job j, denoted 
by Tactual (j) is the actual total time spent by the job in the system from its arrival time 
till it leaves the system (completely finishes). 

Tactualij) — Tjtnish ij) - Tarrivalij) (2) 

where Tfjnzshij) and Tamvaiij) respectively denotes the finish time (time at which job 
j completes its execution) and arrival time for job j. 

Note that the actual execution time for job can be computed only after the job fin¬ 
ishes. 

Definition 2.4. A job j is said to achieve speed up iff Texpij) — Tactuaiij) > 0 and is 
said to be slowed down Texpij) - Tactuaiij) < 0- 

Note that the jobs for which their expected execution time is equal to their ac¬ 
tual execution time, are neither speeded up nor slowed down. If first come first serve 
scheduling would have been used, then actual execution time will be equal to expected 
execution time for all of the jobs and thus, all the jobs will neither be speeded up nor 
slowed down. Depending on the values of expected and actual execution time we can 
say whether a specific job is speeded up or slowed down. 

We dynamically label some of the arriving jobs as urgent (i.e. they are requesting 
speed up). The objective of Speed Up problem is to speed up all the jobs that are re¬ 
questing for speed up without penalizing more than necessary to the rest of the jobs 
present in the queue, while keeping the mean wait time low for all the jobs. It might 
not be possible to achieve speed up for all jobs which requested for it. For example, 
consider a scenario where all the jobs present in the queue are requesting for speed 
up, then it is impossible to achieve speed up for all. Also it might be the case that in the 
process of speeding up of some of the urgent jobs, even some of the jobs that requested 
for speed up were slowed down. This case may arise when an urgent job having high 
processing time is accelerated ahead of another urgent job in the queue. 

Consider a queue of jobs as < ji, j 2 , ja,. ,ji > present in the system at some point 

of time t where I is the queue length at t. Suppose job ji for some 1 < i < / has requested 
for speed up. In order to achieve speed up for job ji, it needs to be accelerated to the 
server. This acceleration will have no impact on all the jobs jk where i + 1 < k < 1. 
But all jobs jp such that 1 < p < i — 1 ahead of ji will suffer a delay equal to d(ji) 
(duration time of job ji). Let there be an urgent job jk for some fee [l,i — 1] which is 
temporarily delayed because of the acceleration of ji, but it is possible that the job jk 
later got accelerated so that eventually it achieved speed up despite of facing an initial 
delay. Therefore it is not always the case, that a delayed job will be slowed down by the 
time it completes its execution. Thus, the priority/importance of any job at any point of 
time t dynamically changes depending on how much it has been delayed till t because 
of acceleration of other jobs. The scheduling decisions need to be made according to the 
on-line speed up requests posed by the user at run-time. 

Definition 2.5. A job j is said to achieve speed up of x time units iff Texpij) 
Tactuaiij) = ^ r > 0 and is said to be slowed down by z time units iff 

Tactuaiij) Texpij) ~ ^ and z > 0. 

Example 2.6. Consider the jobs arriving as per Table [B Let jobs j/ 2 , j /3 and j /5 
are the jobs which are requesting for speed up. Suppose the order of schedule is 
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Table I. Information of jobs arriving to the system. 


Job Id 

Arrival I'ime 

Job Duration 

jo 

0 

5 

jl 

1 

5 

j'2 

2 

3 

j'3 

3 

4 

ji 

4 

3 

j'5 

5 

5 


Table II. Speed Up/Slowdown information. 


Job Id 

Arrival Time 

Job Duration 

Texp 

'^actual 

Speed Up/Slow Down 

jo 

0 

5 

5 

5 

no speed-up/slow-down 

jl 

1 

5 

9 

24 

slow down by 15 units 

j'2 

2 

3 

11 

15 

slow down by 4 units 

j'3 

3 

4 

14 

11 

speed up by 3 units 

ji 

4 

3 

16 

16 

no speed-up/slow-down 

j'3 

5 

5 

20 

5 

speed up by 15 units 


First job jo is picked by the server for service at time t = 0 . At 
t = 0 since there is no job in the queue therefore, T^xpijo) is 5 equal to its own duration 
time. Thus, job jo is neither speeded up nor slowed down. At t = 1 , job ji arrives 
and it finds that there is no job in the queue waiting for its service and jo is being 
served by the server, therefore T^xp{ji) is 5 + 4 (remaining service time of jo) that is 
9 time units. At t = 2, job j /2 finds job ji waiting in the queue, and thus rexp(j' 2 ) is 3 
(d(j/ 2 )) + 5 (d(ji)) + 3 (remaining service time of jo) that is 11 time units. Similarly, 
expected execution time values for j/ 3 , j^ and j /5 are respectively 14, 16 and 20 units. 
At t = 5, when jo finishes, urgent joh j /5 is speeded up to the server for service. Then 
all the jobs from ji to ji will he delayed hy 5 time units including the urgent jobs 
j /2 and j/ 3 . The job j /5 finishes at t = 10. The actual execution time for j /5 is thus, 
5 units. Since Texpijf^) > Tactuai{jib), j's is said to achieve speed up of 20-5 = 15 
units. Now the delayed job j /3 is selected for service after j /5 at t = 10. Then j /3 will 
finish at t = 14. The TactuaiU' 3 ) is thus 14 — 3 = 11 time units. However, the expected 
execution time for j /3 was 14 units. Thus even after facing the initial delay, j /3 is 
successful in achieving speed up of 3 time units. Now job j /2 will be scheduled next 
and will finish at t = 17. The actual execution time for j /2 is thus 17-2 = 15 units. 
However, the expected execution time for j /2 is 11 units. Thus, job j /2 is slowed down 
by 15-11 = 4 units despite of requesting for speed up. Similarly job ji followed by 
ji executes. The information about expected execution time, actual execution time, 
speed up/slow down for this example as per the supposed schedule is shown in Table IHl 


2.1. Problem Classification 

The problem of speed up can be classified into following two categories depending upon 

whether the urgent jobs are requesting for an exact amount of speed up or not: 

(1) Speed Up with no constraints: In this problem, some of the jobs present in the 
queue are requesting speed up, however there is no constraint on the amount of 
speed up that they request. The purpose is to achieve speed up for as many urgent 
jobs as possible without unnecessarily slowing down the other jobs while keeping 
the mean wait time low for all the jobs. 

(2) Fine Grained Selective speed up: In this problem, the urgent jobs request speed 
up of some specific time units. The maximum amount of speed up that a job j can 
achieve is clearly equal to the summation of duration times of all the jobs that are 
in front of it at the time of it’s arrival. So each urgent job can request for some 
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percentage of the maximum speed up that it can achieve at that time. Formally, 
amount of speed up requested hy an urgent joh j denoted hy RSU{j) arriving at 
time t, is given hy 

RSUU) = {^)x Y. d{k) (3) 

vkes(t) 

for some pj € (0,100] where RSU{j) stands for Requested Speed Up hy joh j, S{t) 
represents the system state at time t and d{k) is the duration time of joh k. 

The objective of Fine Grained Selective Speed Up is to achieve (almost) exact 
requested speed up for as many urgent jobs as possible without unnecessarily slow¬ 
ing down other jobs while keeping the mean wait time low for all the jobs. 

To summarize, Speed Up problem aims at meeting the following objectives: 

— To maximize the number of urgent jobs which achieve speed up (exact requested 

speed up in case of fine grained selective speed up). 

— Avoid unnecessary slow down for other jobs while accelerating urgent jobs. 

— Reducing the overall mean wait time for all the jobs. 

— Avoid starvation for any of the job (whether urgent or non-urgent). 

3. SPEED UP ALGORITHMS 

We provide implicit techniques where the notion of acceleration is incorporated in 
the priority function. We present following three strategies for addressing speed up 
problem depending on the amount of slow down that can be tolerated for the non¬ 
urgent jobs: 

(1) Urgent Dominant Speed Up (UDSU): The focus of this technique is to accelerate 
the urgent jobs as much as possible without even bothering about the slow down 
caused to other non-urgent jobs while keeping the mean wait time low. This strat¬ 
egy gives very high priority to the urgent jobs and might even starve non-urgent 
jobs in the presence of continuous arrival of urgent jobs. 

(2) Non-Urgent Benevolent Speed Up (NUBSU): This strategy gives high priority to 
the urgent jobs, but at the same time it opportunistically tries to accelerate delayed 
non urgent jobs as long as such acceleration is not a hurdle in achieving speed up 
for rest of the urgent jobs present in the queue at that time. 

(3) Fair Speed Up (FSU): This strategy is fair to all the jobs and does not lead to high 
slow down for any of the job. FSU strategy is neither biased towards urgent jobs 
nor does create starvation for any job while at the same time aiming at reducing 
the overall mean wait time. With this strategy we can still implicitly achieve speed 
up for some jobs. 

The priority function for job j, denoted by P{j) is defined as follows: 

P{j) = TarrivalU) * d{j) (4) 

where TarrivaiU) denotes the arrival time of job j and d{j) denotes it’s duration time. 

The job with lower value of priority function is preferred. The idea is to give pref¬ 
erence to small size jobs (to improve mean wait time) but this preference takes into 
consideration the arrival time of job to avoid starvation. The priority function ensures 
no starvation for jobs while still preferring shorter jobs (to improve mean wait time) 
because for each job j, since job sizes are finite, there exists certain arrival time (for 
some other job jf) TarrivaiU') > Tarrivai{j) such that all the jobs j/ after j (irrespective 
of their sizes) will have higher priority value than j. Thus, each job j would eventually 
be served. 
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Though the priority function has some commonality with existing scheduling tech¬ 
niques, but we study it in context of speed up and slow down. Further, we extend our 
priority function for different applications (refer Section 5 and 6). 

Definition^.! {Candidate Job). Let J{t) be any set comprising of jobs present in 
the system at time t including the jobs that arrive at t. A job j is said to be Candidate 
with respect to set J{t) for some function p{j) denoted by Cj^p{t), iff j € J{t) and p{j) 
is minimum among all the jobs present in J{t). If there are multiple such jobs with 
minimum function value, then Cj^p{t) is the one with shortest duration time among 
them. 

At any point of time t, let Jurgent{t) and Jnonurgent{t) respectively denote the set of 
all the urgent (jobs requesting for speed up) and non-urgent jobs waiting in the queue 
for service. 

3.1. Speed Up with no constraints 

In this problem, some of the jobs present in the queue are requesting for speed up. 
However, there is no constraint on the amount of speed up requested. The objective is 
to achieve speed up for urgent jobs (actual execution time should be less than expected 
execution time), without penalizing more than necessary to the rest of the jobs. 


ALGORITHM 1: UDSU algorithm. 

Input: Input queue of jobs (queue size > 1) at time t. 
Output: Next job to be executed. 

1: j v- null. 

2: if Jurgent{t) = (f> then 

3: j •«- 

// Tarrival{j) * d(j) iS minimum and j G Jjionurgejitit) 

4: else 

5: j ^ Cj„,.g,,„t,p{t) 

// Tarrivalij) * d{j) is minimum and j G Jurgent{t) 

6: end if 
7: return j 


3.1.1. Urgent Dominant Speed Up Algorithm (UDSU). UDSU Algorithm accelerates the ur¬ 
gent jobs to the server based on the priority value of it. The urgent job with least P 
(as defined in|4ll function value is chosen (in case of ties the one with shorter duration 
time). If there is no urgent job present in the queue, then the non-urgent job with least 
P priority function value is chosen. The notion of acceleration is incorporated in the 
priority function. This algorithm completely ignores the non-urgent jobs in the pres¬ 
ence of urgent jobs and focuses on accelerating as many urgent jobs as possible, thus 
referred to as Urgent Dominant Speed Up algorithm. Note that this algorithm will 
cause the other non-urgent jobs to have arbitrarily high slow down or can even create 
starvation for them in the presence of continuous arrival of urgent jobs. 

To implement this algorithm, we need to maintain two separate priority queues 
one for the urgent and the other one for non-urgent jobs, based on their priority value. 
The time complexity of UDSU algorithm to pick up the next job at time t is 0{log{ 
\Jurgentit)\) + log{ \ J nonurgent (t)]) ) and is quite efficient as compared to all the three 
existing algorithms which have time complexity 0{\Jurgentit)\ + \Jnonurgent{t)\). 
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3.1.2. Non-Urgent Benevolent Speed Up Algorithm (NUBSU). This algorithm gives high pri¬ 
ority to the jobs which are requesting speed up, hut is not unfair towards other jobs. It 
optimistically tries to accelerate other jobs as long as such acceleration is not posing 
any hurdle in achieving speed up for existing urgent jobs. 

Definition 3.2 {Opportunistically forwardable job). A job j is said to be opportunis¬ 
tically forwardable at time t if j e Jnonurgent{t) and Vfc e Jurgent{t), 

[Twait{k,t) + d{j) + ^ d{kl) -I- drem{js)] < Texp{k) (5) 

P(kl)<P(k) 

where fc/ e Jurgent{t) such that P{kf) < P{k) , Tu,ait{k, t) denotes the waiting time of job 
k till time t and drem{js) is the remaining duration time of job js getting serviced at 
server at t if any else zero. 


ALGORITHM 2: NUBSU algorithm 

Input: Input queue of jobs (queue size > 1) at time t. 

Output: Next job to be executed. 

1 : j <r- null. 

2: if J nonurgent ft) = (f) ov ,p (i) is not Opportunistically forwardable at t then 

3: j 

H Taxrivallj) * d{j) iS miuimum and j ^ Jurgentft) 

4: else 

5: j P- 

H Tarrival{j) * d{j) is minimum and j ^ dnonurgent{t) 

6: end if 
7: return j 


A non urgent job j nonurgent is opportunistically forwardable at time t, if it’s accel¬ 
eration to the server for service will not affect the existing urgent jobs (present in the 
queue) in achieving their speed up at t. That is, existing urgent jobs can achieve speed 
up even after the acceleration of non-urgent job j nonurgent to the server for service. In 
such case, each urgent job jurgent has to further wait for all the other urgent jobs which 
have lower P{j) function value (higher preference) than jurgent plus the duration time 
of jnonurgent since jnonurgent is Opportunistically forwardable and thus, will be acceler¬ 
ated for service before jurgent ■ Thus, for jurgent to achieve speed up, the time that it 
has to further wait (including it’s own duration time) plus the time that it has already 
spent waiting in queue till now should be less than it’s expected execution time. 

NUBSU Algorithm gives high priority to the urgent jobs, but is still optimistic to¬ 
wards non urgent jobs in a sense that it accelerates them whenever such acceleration 
does not pose any hurdle for urgent jobs in achieving their speed up. The important 
question is does acceleration of opportunistically forwardable job always ensure that 
the other urgent jobs will achieve speed up? The answer is no. Our algorithms are on¬ 
line and we do not have any information about the jobs arriving in future. So it may 
happen that at time t, a non-urgent job jnonurgent is opportunistically forwardable and 
thus is forwarded to the server for service, but during the execution of jnonurgent, a 
bunch of urgent jobs arrives in the queue say at time t + St and some of which are hav¬ 
ing their Priority function value lower than some existing urgent job jurgent (might be 
because of very low duration time). So jurgent for which summation of duration times 
of all the urgent jobs having lower Priority function value plus duration of jnonurgent 
was less than Texp{jurgent) at time t, can now become greater than T^xpijurgent) at time 
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t + St (because of arrival of some urgent jobs which would have been executed before 
jurgent because of their low P function value). Thus, jurgent cannot achieve speed up. 
This situation is analogous to non preemptive SJF scheduling where recently arrived 
short job has to wait for currently executing long job to finish (since the algorithm 
is on-line and non preemptive). To measure the probability of such events, we define 
the notion of successful!unsuccessful opportunistically forwarded non urgent jobs. A 
successful opportunistically forwarded non-urgent job jnonurgent is the one whose ac¬ 
celeration to the server for service does not impact any other existing urgent job in 
achieving speed up by the time it complete its execution. Similarly, unsuccessful op¬ 
portunistically forwarded job jnonurgent is the one whose acceleration prevented some 
of the existing urgent jobs in achieving their speed up. 

Definition 3.3 {Degree of Non Urgent Job Impact (DNUJI)). Degree of Non Urgent 
Job Impact (DNUJI) for NUBSU algorithm is defined as the ratio of number of 
unsuccessful opportunistically forwarded jobs to the total number of opportunistically 
forwarded jobs. 


DNUJI ^ui^berOfUnsuccessfulOpportunisticallyForwardedJobs 
TotalNumberO f Opportunistically ForwardedJ obs 

To implement NUBSU algorithm, we maintain a priority queue for non urgent jobs 
based on P function value. For urgent jobs, we maintain a AVL tree based on their P 
function value. To check whether is opportunistically forwardable, we 

need to traverse the AVL tree in increasing order of P function values by doing an 
in-order traversal of tree and checking for each urgent job k, whether 

d{kt^ -\- drem{j ^ 'Fexp{k') (I) 

where k/ e Jurgent{t) such that P{k/) < P{k) and drem(js) denotes the remaining dura¬ 
tion time of currently executing job js- The time complexity for obtaining C Jurgent,pit) 
and is 0{log{ \Jurgent{t)\) + log{ \Jnonurgent{t)\) )• However, the time 

complexity of determining whether is opportunistically forwardable is 

0{ \Jurgent{t)\) since we need to traverse the whole AVL tree containing urgent jobs at 
time t. Thus, the overall time complexity of NUBSU Algorithm for picking up the next 

job at time t is 0{log{ \ Jurgent{t)\) + log{ \Jnonurgent{t)\) + \ Jurgent{t)\ ). TMs algorithm 

is computationally more expensive than UDSU algorithm. 

3.1.3. Fair Speed Up Algorithm (FSU). This algorithm is not biased to any specific kind 
of job. It chooses the job which is having least P function value among all the jobs 
present in the queue (including both urgent and non-urgent jobs) to schedule next. If 
there are multiple such jobs, then the job with smaller duration time is selected. Note 
that, both UDSU as well as NUBSU algorithms can create starvation for non-urgent 
jobs in the presence of continuous arrival of urgent jobs. However, FSU provides less 
scope of starvation for any of the job. Each job would eventually be served because the 
next job will be the one having least P function value among all the jobs present in 
the queue. Therefore, a job waiting for too long will have low value of arrival time as 
compared to other jobs which arrived after it, and since job sizes are finite, thus a time 
t will always exist such that its P function value is minimum among all the jobs in 
the system. The major drawback of this algorithm is that it does not serve the purpose 
of achieving speed up for as many urgent jobs as possible, but we can still implicitly 
achieve speed up for some of the jobs. The benefit of this algorithm is that it does not 
starve any of the job along with improving the overall mean wait time. 

To implement this algorithm, we need only one priority queue containing all the 
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jobs present in the queue based on their P priority value. Thus, the time complexity to 
pick up the next joh for FSU algorithm is 0{log I ) where I is the length of the queue. 


3.2. Fine Grained Seiective speed Up 

In this speed up problem, each of the urgent joh requests for a specific amount of 
speed up denoted by RSU{j) which from eq.|3]is defined as 


RSU{j) = ^ X ( 


E 


d{k)) 


vfeeS(T„, 


lU)) 


for some pj G (0,100 
By the definition 


2.2\ o{RSU{j) can also be written as 

RSU{3) = ^ X (re,p(j) - d{j) - dreM) 


( 8 ) 


(9) 


Definition 3.4. An urgent job jurgent is said to be successfully speeded up iff 


(Rexp^jurgent) Ractuali^jurgent)) P RSUi^jurgent) 


( 10 ) 


The objective of Fine Grained Selective Speed Up is to maximize the number of suc¬ 
cessfully speeded up jobs without unnecessarily slowing down other jobs while keeping 
the mean wait time low. 


Definition 3.5. The current speed up for an urgent job jurgent at time t, denoted by 
CSU(jurgent, t) is defined as follows: 


DSU(jurgent, l) — Rexp(jurgent) ('Rwait(jurgent,l) d(jurgent) drem Us)) 


where 


( 11 ) 


'-ll’wait(jurgent, i) — I ^arrival (jurgent) 


( 12 ) 


The current speed up at point of time t for job j represents the amount of speed up 
that it will achieve if it would have been accelerated next to the server for service just 
after the execution of job currently being served. 

The priority of an urgent job j at time t denoted by PurgentU, t) is defined as: 


Rurgent (j, t) = CSU(j, t) - RSU(j) (13) 

Using eq. [HITT] and [121 we get 


Rurgent(j,l) — [Rarrival(j) P (1 * ('Rexp(j) d(j) drem(js))\ t (14) 

However, the time t at which we will calculate the priority value and drem(js) re¬ 
mains the same for all urgent jobs, so value of t and drem(js) won’t affect in comparing 
the priorities and thus, we can ignore it. Therefore, 


Rurgent (j ) — Rarrival U) + (1 - Pjim * [TexpU) - d(j)] (15) 

The function Rurgent is used as priority function among urgent jobs (lower the value 
of function, higher the importance/preference of it). It can be inferred from ea 1 151 that 
a job with low arrival time requesting for high percentage of speed up having low 
expected execution time is preferred. Note that, an urgent job j will be successfully 
speeded up iff CSU(j, t) > RSU (j) where t is the time at which job j was accelerated to 
the server for service. For non-urgent jobs, we use previous priority function P defined 
in eq.|4]for computing their priority. 

In order to present the three strategies for speed up for this case, we present a 
Generic Probabilistic Speed Up (GPSU) algorithm which based on the value of proba¬ 
bility p can mimic UDSU, NUBSU and FSU algorithm. 
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ALGORITHM 3: GPSU Algorithm. 

Input: Input queue of jobs (queue size > 1) at time t and probability parameter p 
Output: Next job to be executed. 

1: j i- null. 

2: if Jurgentit) = <j> then 

3: j ^ 

4: return j 

5: end if 

6l if Jnonurgent{t') — ^ then 
j ^ ^.^urgent :^urgent 

8: return j 

9: end if 

10: Generate a random number x between 0 to 1 inclusive. 

11: if a: < p then 

j ^ ^Jurgent.Purgent^^') 

H Purgent(.j^ iS minimum and_j ^ Jurgentij^ 

13: else 
14: j •(- 

H Taj.Tr.ivali.j') * d(j) IS minimum and j G Jnonurgentit^ 

15: end if 
16: return j 


3.2.1. Generic Probabilistic Speed Up Algorithm (GPSU). GPSU Algorithm takes a proba¬ 
bility parameter p which is a measure of bias between urgent and non-urgent jobs. 
Higher the value of p, higher is the dominance of urgent jobs over non-urgent jobs. 
GPSU algorithm tries to mimic UDSU, NUBSU and FSU algorithm based on proba¬ 
bility parameter p. 

For p = 1, GPSU algorithm mimics the nature of UDSU algorithm. The algorithm 
will completely ignore the non urgent jobs in the presence of urgent jobs and will lead 
to their starvation in the presence of frequent arrival of urgent jobs. For higher values 
of p < 1 , the algorithm will pick urgent jobs with very high probability, however at 
the same time, the algorithm will not create starvation for non urgent jobs since the 
slowed down non urgent jobs will be picked up by their priority function (low arrival 
time resulting in low priority function value) with probability (1-p) thereby mimicking 
NUBSU algorithm. For p proportional to the fraction of total jobs requesting for speed 
up (urgent jobs), the algorithm will behave as FSU algorithm since we are giving equal 
chance to both urgent as well as non-urgent jobs. 

The algorithm can be implemented by using two separate priority queues one for 
the urgent jobs (based on PurgentU) function value as per eq. [15]) and the other one 
for non-urgent jobs (based on P function value defined in eq. jUt. The time complexity 
of this algorithm is thus 0{log{ \ Jurgent{t)\) + log{ \ J„onurgentit)\) ) to pick up the next 
job at time t and is computationally efficient. It is important to note that GPSU algo¬ 
rithm can also be used for Speed up with no constraints scenario, with only one minor 
change that instead of using Pur gent, use earlier defined P priority function defined in 
eq. m in the GPSU algorithm. The NUBSU algorithm had high time complexity and 
thus, GPSU algorithm can be used to mimic its nature by setting the probability pa¬ 
rameter p. 

Value of p to be used in GPSU algorithm depends on the required prefer¬ 
ence/dominance of urgent over non-urgent jobs. If jobs requesting for speed up are 
extremely critical and urgent, then p should be 1 or very close to it. However, if jobs 
requesting for speed up are not much critical and high slow down for any of the job 
might have severe consequences then in that case value of p can be set somewhere 
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Table 111. p = l.l: Percentage of urgent jobs that achieved speed up. 


Urgent jobs 

MPF 

MPF-SD 

MinPJ:'’ 

UDSU 

NUBSU 

T'SU 

0 % 

- 

- 

- 

- 

- 

- 

10 % 

97% 

94% 

97% 

97% 

94% 

84% 

20 % 

55% 

913% 

55% 

95.5% 

513% 

84% 

30% 

94.67% 

93.3% 

94.33%. 

95% 

90% 

84.33% 

40% 

933% 

933% 

94.25% 

94.5% 

89.75% 

85.25% 

50% 

94.2% 

94.6% 

95.2% 

95% 

90.6% 

86 .6% 

60% 

93.67% 

92.83% 

93.83% 

94.5% 

90.5% 

86.33% 

70% 

92.71% 

52% 

93.86% 

53% 

89.75% 

87% 

80% 

91.62% 

90.62% 

93% 

93% 

90.88% 

87.25% 

90% 

89.78% 

89.67% 

90.89% 

90.56% 

90.56% 

87.56% 

100 % 

- 

- 

- 

87.3% 

87.3% 

87.3% 


between 0.6 - 0.9. Value of probability parameter is thus dependent on the requirement 
of the environment based on the amount of slow down that can be tolerated and the 
amount of speed up which is intended/desired by the system. 

4. EXPERIMENTS AND RESULTS 

We used a discrete event based simulation tool for the purpose of comparison anal¬ 
ysis and evaluation of our speed up algorithms. We assumed that jobs are arriving in 
Poisson distribution. The total number of jobs we considered for the experiments were 
10 thousand. The service time of jobs were assumed to be in exponential distribution. 
We conducted experiments for three different values of system load p (A/p) equal to 

1.1, 1.3 and 1.5. Since the nature of results obtained were found similar for different p 
values, therefore we present here the results only for p value equal to 1.1. We labeled 
some of the arriving jobs among total jobs as urgent and conducted experiments by 
varying proportion of urgent jobs (jobs requesting for speed up) from 0% to 100%. 
Value of p used is 0.04 and A varies depending on the value of p. 


4.1. Speed Up with no constraints 

4.1.1. Percentage of urgent jobs that achieved speed up. The Table Hill gives the percentage 
of urgent jobs (jobs requesting for speed up) that achieved speed up for different speed 
up algorithms based on the proportion of urgent jobs for system load p = 1.1. UDSU al¬ 
gorithm is successful in achieving near about similar (sometimes even higher) amount 
of speed up as compared to existing speed up algorithms MPF and MinPF. The UDSU 
algorithm outperforms MPF-SD in terms of achieved speed up for majority of urgent 
job proportion. It is interesting to note that the performance of NUBSU algorithm 
is lower than UDSU algorithm and other existing speed up algorithms in terms of 
achieved speed up validating our idea that the acceleration of opportunistically for¬ 
wardable job does not always ensure that every other existing urgent job present in 
the queue will achieve speed up and thus, such acceleration may even slow down the 
urgent jobs. But still, NUBSU algorithm speeds up large number of urgent jobs. The 
percentage of speeded up urgent jobs is lowest for FSU algorithm since it is fair to 
all and is not biased towards urgent jobs. When all the jobs present in the queue are 
requesting for speed up (100% case), then all the three speed up (UDSU, NUBSU and 
FSU) algorithm behaves as same. As the percentage of urgent jobs requesting for speed 
up increases, achieved speed up decreases. 

4.1.2. Percentage of slowed down non urgent Jobs. Table HVl shows that our implicit speed 
up algorithms (UDSU, NUBSU and FSU) performs better than MPF, MPF-SD and 
MinPF algorithms in terms of slow down caused to the other non-urgent jobs present 
in the queue. The reason is that MPF, MPF-SD and MinPF algorithms do not consider 
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Table IV. p = l.l: Percentage of slowed down non-urgent jobs. 


Urgent jobs 

MPF 

MPF-SD 

MinPb’ 

UDSU 

NUBSU 

T'SU 

0 % 

- 

- 

- 

8.8% 

8.8% 

8.8% 

10 % 

65.78% 

8.56% 

89.11% 

10.89% 

9.56% 

8.44% 

20 % 

67% 

15.25% 

94.62% 

13.38% 

10.25% 

8.12% 

30% 

71.86% 

24.57% 

95.67% 

16.29% 

13.14% 

7.71% 

4(5% 

68.67% 

3T% 

95.83% 

22% 

153% 

7.83% 

50% 

75% 

39.8% 

96.2% 

27% 

22.8% 

8.4% 

60% 

80% 

51.5% 

96.75% 

35.25% 

35% 

8% 

70% 

82% 

39% 

96% 

3B% 

41.67% 

~S% 

80% 

84.5% 

60% 

96.5% 

60% 

59% 

9% 

90% 

88 % 

82% 

97.4% 

86 % 

90% 

12% 

100 % 

- 

- 

- 

- 

- 

- 


Table V. p = l.l: Mean Walt Time (in time units). 


Urgent jobs 

MPF 

MPF-SD 

MinPF 

UDSU 

NUBSU 

FSU 

0 % 

- 

- 

- 

473 

473 

473 

10 % 

1742 

1575 

1713 

495 

535 

473 

20 % 

1738 

1459 

1711 

536 

602 

473 

30% 

1715 

1372 

1694 

558 

668 

473 

40% 

1709 

1288 

1692 

584 

759 

473 

50% 

1747 

1164 

1675 

633 

779 

473 

60% 

1705 

1137 

1669 

738 

848 

473 

70% 

1672 

1069 

1625 

823 

876 

473 

80% 

1670 

1073 

1591 

925 

897 

473 

90% 

1562 

1259 

1548 

1111 

1111 

473 

100 % 

- 

- 

- 

473 

473 

473 


about the slow down of other non-urgent jobs and greedily aim at achieving high speed 
up for urgent jobs using location table and swapping of jobs. However, in our speed up 
algorithms,the non-urgent jobs are served using the priority function which takes into 
consideration the arrival time (to avoid high wait time) as well as duration time (to 
improve the mean wait time) which avoids high slow down for non urgent jobs. Note 
that NUBSU algorithm performs better than UDSU algorithm in terms of slow down 
since NUBSU algorithm whenever it finds some non-urgent job to be opportunistically 
forwardable, it accelerates it to the server resulting in less percentage of slow downs 
for non-urgent jobs as compared to UDSU algorithm. It is also observed that as the 
proportion of urgent jobs increases, the slow down caused to non-urgent jobs also in¬ 
creases. FSU algorithm provides least amount of slow down to non urgent jobs since 
FSU is fair to all and is not biased towards any specific job. 

4.1.3. Mean Wait Time. Table |V| shows the overall mean wait time for all the jobs 
present in the queue for different urgent job proportion. The mean wait time for FSU 
algorithm is lowest owing to its unbiased nature and it remains unaffected with re¬ 
spect to the proportion of urgent jobs since the algorithm is independent of it. Our 
speed up algorithms outperforms existing position based speed up algorithms (MPF, 
MinPF and MPF-SD) in terms of overall mean wait time owing to the design of the 
priority function which prefers job with shorter duration time (resulting in low mean 
wait time). 

4. 1.4. 95 percentile metric for wait time. Table IVllindicates the 95 percentile metric of wait 
time for different speed up algorithms. Experimental results show that MinPF algo¬ 
rithm provides the least maximum wait time among all the existing algorithms. The 
reason is MinPF algorithm swap jobs with minimum distance. Our implicit speed up 
algorithms (UDSU, NUBSU and FSU) outperforms the existing speed up algorithms 
in terms of 95 percentile metric owing to the design of the priority function. 
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Table VI. p = l.l: 95 percentile metric/maximum for wait time (in time units). 


Urgent jobs 

MPF 

MPF-SD 

MinPb' 

ODST] 

NUBSU 

T5U 

0 % 

- 

- 

- 

3269/14002 

3269/14002 

3269 / 14002 

10 % 

4018/7817 

3857/10160 

3911/4125 

3421 / 14218 

3449 / 14008 

3269 / 14002 

20 % 

5085/ 11657 

3985 / 13053 

4136 / 4263 

3661 / 14533 

3307 / 14394 

3269/14002 

30% 

6119/15324 

4505/12132 

4491 / 4644 

3875 / 14825 

3254 / 14498 

3269/14002 

40% 

6770 / 15854 

4929 / 14302 

5423/6011 

3658 / 16840 

3503 / 14829 

3269/14002 

50% 

8257 / 17293 

5303 /17525 

6049 / 6959 

3822 / 17860 

3660 / 16049 

3269 / 14002 

60% 

8970 / 14967 

5495/19537 

7046 / 7695 

4873 / 16533 

4002 / 16452 

3269 / 14002 

70% 

9751/21812 

6020 / 19069 

8321 / 9905 

5707 / 17281 

4106 / 16798 

3269 / 14002 

80% 

11355/22886 

6526/21901 

9638/ 12983 

6485 / 17795 

4750 / 17795 

3269 / 14002 

90% 

9615/21751 

7308 / 19959 

9875 / 18488 

10041 / 20491 

9872 / 20491 

3269 / 14002 

100 % 

- 

- 

- 

3269 / 14002 

3269 / 14002 

3269 / 14002 



Fig. 1. Degree of non-urgent job impact (DNUJI) for NUBSU algorithm for different values of system load. 

4.1.5. DNUJI for NUBSU algorithm. Figure [I] shows the plot of DNUJI for NUBSU al¬ 
gorithm based on proportion of urgent johs for different values of system load p. The 
non-zero value of DNUJI validates our idea that acceleration of opportunistically for¬ 
wardable joh do not always ensure that every other urgent job would achieve speed up. 
But, experimentally it still happens even less than 5 times out of 100. 

4.2. Fine Grained Seiective Speed Up 

In order to consider Fine Grained Selective Speed Up scenario, among all the ur¬ 
gent jobs (requesting for speed up) we assigned each of them random percentage value 
pj G (0,100] corresponding to which their RSU will be calculated. Note that existing 
MPF, MPF-SD and MinPF algorithms are not applicable for cases like Fine Grained 
Selective Speed Up where each job j requests for some specific pj% of max speed up 
that it can achieve. Therefore, we present here the results for our GPSU algorithm for 
different values of probability parameter p. 

4.2.1. Percentage of successfully speeded up urgent Jobs. Table Ivni shows the percentage of 
successfully speeded up urgent jobs (jobs for which achieved speed up >= Requested 
speed up ) for GPSU algorithm for different values of probability parameter p. For 
p — I, maximum successful speed ups were obtained. The percentage of successful 
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Table VII. p = l.l: Percentage of successfully speeded up urgent 
jobs for GPSU Algorithm. 


Urgent jobs 

p=l 

II 

o 

II 

o 

bo 

II 

p 

Ui 

0 % 

- 

- 

- 

- 

10 % 

94% 

94% 

94% 

92% 

20 % 

92% 

91.5% 

91% 

89% 

30% 

92% 

91.33% 

91% 

90% 

40% 

91.75% 

90.5% 

89.25% 

88.25% 

50% 

92.6% 

91.8% 

90.8% 

82.8% 


90.17% 

87.83% 

85.17% 

~7Wc 

70% 

87.57% 

85.57% 

82.43% 

60.43% 

80% 

81.5% 

79.12% 

36.75% 

2 .2% 

90% 

57.11% 

5% 

3.3% 

2 .11% 

100 % 

2.2% 

2.2% 

2 .2% 

2 .2% 


Table VIII. p = l.l: Percentage of slowed down non-urgent jobs for 
GPSU Algorithm. 


Urgent jobs 

p=l 

II 

o 

II 

o 

bo 

II 

p 

0 % 

8 .8% 

8 .8% 

8.8% 

8.8% 

10 % 

10.89% 

10.89% 

10.78% 

10.78% 

20 % 

13.38% 

13.38% 

13.25% 

13% 

30% 

16.29% 

15.86% 

15.43% 

15.43% 

40% 

22 % 

21.33% 

20.17% 

19.67% 

50% 

27% 

25.8% 

24.4% 

23.4% 

60% 

35.25% 

35% 

32.25% 

28% 

70% 

46% 

41.33% 

37.33% 

28.67% 

80% 

64% 

55% 

32% 

5.5% 

90% 

90% 

46% 

9% 

2% 

100 % 

- 

- 

- 

- 


Table IX. p = l.l: Overall mean wait time for all the jobs for 
GPSU Algorithm. 


Urgentjobs 

p=l 

II 

o 

p=0.8 

II 

p 

0 % 

473 

473 

473 

473 

10 % 

495 

495 

495 

495 

20 % 

537 

536 

530 

537 

30% 

560 

559 

560 

559 

40% 

589 

588 

591 

587 

50% 

645 

642 

644 

644 

60% 

760 

749 

752 

760 

70% 

868 

850 

847 

947 

80% 

1039 

984 

1129 

1711 

90% 

1532 

1441 

1728 

1751 

100 % 

1717 

1717 

1717 

1717 


speed ups decreases as probability parameter p decreases, since the tendency to favor 
urgent jobs decreases with decrease inp. It is also experimentally observed, that as the 
proportion of urgent jobs requesting for speed up increases, percentage of successful 
speed ups decreases. 

4.2.2. Percentage of slowed down non urgent Jobs. Table IVIIII shows the percentage of 
slowed down non-urgent jobs for GPSU Algorithm for different values of probability 
parameter p. Higher the probability parameter, higher is the percentage of slowed 
down non-urgent jobs since as p increases, tendency to serve non-urgent jobs decreases 
which leads to their slow down. Thus, the percentage of slowed down jobs which are 
not requesting for speed up is least for p = 0.7. 
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Table X. Sample process log records obtained using pacct utility in linux. 


command 

version 

utime 

systime 

etime 

uid 

gid 

mem 

char 

pid 

ppid 

finish time 

cclplus 

v3 

17.00 

1.00 

19.00 

1000 

1000 

31632.00 

0.00 

27379 

27378 

Sun Dec 15 16:47:50 2013 

a.out 

v3 

16.00 

0.00 

16.00 

Tow 

TfiOO" 

7988.00 

0.00 

27389 

27324 

Sun Dec 15 16:47:51 2013 

Id 

v3 

5.00 

0.00 

6.00 

1000 

1000 

10512.00 

0.00 

27554 

27553 

Sun Dec 15 16:51:02 2013 


Table XI. Process logs dataset: Percentage of urgent jobs that achieved speed up for Speed 
up with no constrains scenario. 


Urgent jobs 

MPP 

MPF-SD 

MinFJ*' 

UDSU 

NUBSU 

TI'SD 

0 % 

- 

- 

- 

- 

- 

- 

10 % 

98.99% 

97.98% 

99.8% 

98.99% 

97.98% 

82.83% 

20 % 

98.99% 

98.48% 

99.49% 

98.99% 

95.96% 

79.29% 

30% 

98.32% 

98.99% 

99.33% 

99.2% 

94.28% 

79.12% 

40% 

99.14% 

99.75% 

99.49%. 

99.24% 

95.71% 

85.75% 

50% 

98.79% 

99.39% 

98.79% 

99.39% 

96.16% 

80.61% 

60% 

98.65% 

99.16% 

98.65% 

99.1%. 

97.47% 

80.64% 

70% 

98.56% 

98.99% 

98.56% 

98.67%. 

96.97% 

80.95% 

80% 

98.48% 

99.12% 

98.48% 

96.09% 

95.96% 

81.44% 

90% 

98.32% 

98.32% 

98.2% 

91.69% 

90.69% 

81.14% 

100 % 

- 

- 

- 

81.33% 

81.33% 

81.33% 


Table XII. Process logs dataset: Percentage of slowed down non-urgent jobs for speed up with 
no constraints scenario. 


Urgent jobs 

MPF 

MPF-SD 

MinPF 

UDSU 

NUBSU 

FSU 

0 % 

- 

- 

- 

18.16%. 

18.16%. 

18.16%. 

10 % 

84.75% 

9.08% 

97.98% 

20.63% 

20.29% 

18.27%. 

20 % 

81.2% 

17.78% 

98.87% 

22.45% 

20.3% 

17.65% 

30% 

79.25% 

25.94% 

99.33% 

25.5% 

22.48% 

17.15% 

40% 

68.57% 

34.12% 

99.49% 

33.61% 

23.87% 

12.67% 

50% 

71.77% 

45.16% 

99.6% 

40.52%. 

29.64%. 

17.14%. 

60% 

70.78% 

54.66% 

98.79% 

52.53%. 

51.21%. 

17.38%. 

70% 

93.96% 

79.8% 

99.7% 

78.52% 

72.48% 

17.45% 

80% 

80.4% 

86 .88% 

99.73% 

86.93 

83.92% 

18.59% 

90% 

93% 

87% 

99.8% 

93% 

93% 

17% 

100 % 

- 

- 

- 

- 

- 

- 


4.2.3. Mean Wait time. Table HXl shows the overall mean wait time (in time units) for 
all the jobs for GPSU Algorithm for different probability values. 

4.3. Experimentation on process logs dataset 

Apart from conducting experiments on M/M/1 queueing system, we also conducted 
experiments on real process logs obtained using Process Accounting utility in linux. 
For the purpose of gathering logs, we executed different types of user programs (dy¬ 
namic programming, recursion, modular exponentiation, binary search , adhoc etc.) 
randomnly in poisson fashion using shell script on randomnly generated datasets at 
run time. The RAM of the system is 3GB and processor used is 2.4 GHz. Using pacct 
utility in linux, we collected process logs for about lOK executed processes which in¬ 
volves both user as well as kernel processes. The minimum and maximum cpu burst 
time obtained in the logs were 1 and 185 units respectively. Some of the sample log 
records are shown in table E where command - command name, version - version of 
acct file, utime - user time, systime - system time, etime - elapsed time, uid - user id, 
gid - group id, mem - memory usage, char - number of characters transferred on in¬ 
put/output, pid - process id, ppid - parent pid, finish time - finish time of process. The 
nature of results obtained are similar as compared to M/M/1 system and are presented 
here. 
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Table XIII. Process logs dataset: Mean Wait Time for Speed Up with no constraints 
scenario (in time units). 


Urgent jobs 

MPF 

MPF-SD 

MinPF 

UDSU 

NUBSU 

PSu 

0 % 

- 

- 

- 

1122 

1122 

1122 

10 % 

3507 

3154 

3489 

1185 

1170 

1122 

20 % 

3337 

2765 

3528 

1334 

1251 

1122 

30% 

3573 

2752 

3606 

1516 

1425 

1122 

40^ 

3484 

2556 

3452 

1555 

1525 

1122 

50% 

3420 

2518 

3485 

1735 

1631 

1122 

60% 

3484 

2672 

3573 

2484 

2235 

1122 

70% 

3438 

2716 

3466 

2416 

2315 

1122 

80% 

3215 

2870 

3261 

2186 

2010 

1122 

90% 

3319 

3100 

3335 

1763 

1763 

1122 

100 % 

- 

- 

- 

1122 

1122 

1122 


Table XIV. Process logs dataset: Percentage of successfully 
speeded up urgent jobs for GPSU Algorithm. 


Urgent jobs 

p=l 

II 

o 

II 

o 

bo 

II 

p 

u% 

- 

- 

- 

- 

10 % 

97.98% 

97.98% 

96.97% 

96.97% 

20 % 

97.98% 

97.91% 

96.97% 

96.46% 

30% 

97.93% 

97.64% 

97.63% 

96.64% 

40% 

97.22% 

97.22% 

96.97% 

95.96% 

50% 

95.96% 

93.74% 

93.15% 

92.73% 

60% 

93.43% 

93.77% 

93.1% 

92.42% 

70% 

92.06% 

91.92% 

90.62% 

1 .01% 

80^ 

56.68% 

16.54% 

0.51% 

0.38% 

90% 

1.97% 

1.01% 

0.45% 

0.34% 

100 % 

0.3% 

0.3% 

0.3% 

0.3% 


Table XV. Process logs dataset: Percentage of slowed down non 
urgent jobs for GPSU Algorithm. 


Urgent jobs 

p=l 

II 

o 

II 

o 

bo 

p=0.7 

0 % 

18.16% 

18.16% 

18.16% 

18.16% 

lo^ 

20.63% 

20.63% 

20.21% 

20.21% 

20 % 

22.45% 

22.45% 

22.32% 

22.32% 

30% 

25.5% 

25.4% 

25.36% 

25.21% 

40% 

33.61% 

33.6% 

33.51% 

33.45% 

50% 

40.52% 

41.13% 

40.32% 

40.12% 

60% 

70.53% 

64.48% 

48.61% 

43.83% 

70% 

78.52% 

72.82% 

48.66% 

28.52% 

80% 

86.93% 

51.79% 

17.09% 

1.01% 

90% 

93% 

43% 

2% 

0.2% 

100 % 

- 

- 

- 

- 


Table XVI. Process logs dataset: Overall mean wait time for 
all the jobs for GPSU Algorithm. 


Urgent jobs 

p=l 

II 

o 

II 

o 

bo 

II 

p 

0 % 

1122 

1122 

1122 

1122 

10 % 

1186 

1186 

1185 

1186 

20 % 

1335 

1335 

1334 

1334 

30% 

1518 

1517 

1517 

1518 

40% 

1603 

1596 

1595 

1587 

50% 

1750 

1730 

1720 

1735 

60% 

2571 

2340 

2138 

1980 

70% 

2806 

2596 

2377 

2453 

80% 

3003 

2596 

2847 

3237 

90% 

3196 

2924 

3254 

3293 

100 % 

3297 

3297 

3297 

3297 
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4.4. Discussion 

The authors in BKafeza et al. 200^ and [Kafeza and Karlapalem 2000b) used only 
achieved ratio metric (percentage of urgent jobs that achieve speed up) for evaluat¬ 
ing the effectiveness of their speed up algorithms. In this research, we along with 
achieved ratio metric also consider the amount of slow down for rest of the jobs as a 
result of speed up and overall mean wait time. The experimental results show that our 
proposed speed up algorithm UDSU is able to achieve near about similar amount of 
speed up as compared to the existing speed up algorithms MPF, MPF-SD and MinPF. 
All the three speed up algorithms UDSU, NUBSU and FSU outperforms the existing 
speed up algorithms in terms of slow down caused to the other non-urgent jobs. It is 
because existing speed up algorithms do not provide any remedies for delayed jobs 
and greedily aim at achieving high speed up for urgent jobs. However, our implicit 
speed up priority function takes into consideration the arrival time (while speeding 
up urgent jobs) which avoids high slow down for non-urgent jobs. The overall mean 
wait time is improved dramatically by our speed up algorithms owing to the design 
of priority function as compared to the existing speed up algorithms which are purely 
positional. Further, our algorithms are implicit in nature where the notion of acceler¬ 
ation is incorporated in the priority function which leads to a time efficient solution 
unlike existing algorithms where there is an overhead of swapping of jobs and location 
table. We further presented the GPSU algorithm for Fine Grained Selective Speed Up 
scenario which has not been yet addressed and presented the results for the same. 

We used our experimental methodology to be simulation. The experiments per¬ 
formed involved changing various parameters ( system load, jobs requesting for speed 
up, RSU etc.) and the results shown are the statistical averages of multiple runs. We 
conducted our experiments assuming a M/M/1 queuing system which is widely used 
queueing system for the analysis of re al life situations a nd is a good approximation 
for large number of queueing systems BGross et al. 200^ . The poisson arrival distri¬ 
bution of jobs assumed in experiments is a very good approximation for job arrival in 
real systems. Regarding exponential distribution there is an argument from informa¬ 
tion theory which states that exponential distribution provides the least information 
or highest entropy, and is therefore a reas onable assumption for service time when no 
other data is available BGross et al. 2008ll . Thus, our assumption of M/M/1 system for 
experimental analysis is valid to a greater extent and our results are significant. We 
also achieved similar nature of results using real process logs dataset. Further, we also 
have considered real trace data for experimental analysis presented in later chapters 
of this thesis. 

5. APPLICATIONS: WEB SCHEDULING 

We apply the idea of implicit speed up for the purpose of scheduling of web requests 
arriving at web server. We show how our proposed priority function (defined in eqS]) for 
implicit speed up with little modification, can be applied to web scheduling. We show 
the effectiveness of our algorithms using simulation model on a trace driven workload. 

5.1. Introduction 

Current demand on busy web servers requires them to serve up to thousands of 
clients simultaneously. In such a case, the response time suffered by the client is 
one of the most important factor that determines the web server performance, where 
the response time is defined as the time duration between the time client makes 
the request until the time the client receives the last byte of the file requested. The 
servers cannot afford large delay in response time for users because it might result 
in rejection of request either because of server timeout or due to user abort. The slow 
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respon se times an d difficult navigation are the most common complaints of Internet 
Users | |King 2003] . The scheduling strategies used hy a weh server play a very crucial 
role in determining the response time of users. Now, the question that arises is how to 
improve the response time of the user hut at the same time ensuring no starvation hy 
simply changing the order of servicing of requests arriving at weh server. 

The traditional scheduling strategy used in weh server is Processor-Sharing (PS) 
scheduling in which each of the n incoming requests gets same amount 1/n of the CPU 
time. PS scheduling is unbiased as it gives equal proportion of time to all. 

It is well known from queuing theory that Shortest Remaining Pro¬ 
cessing Time (SRPT) scheduling is an o ptimal sched uling algorithm 
for minimizing the mean response time 
scheduling policies uses size of requested 
Various SRPT based scheduling poli cies |Crovella and Frangioso 19 991, 
I Harchol-Balter et al. 20031. _ ISchroeder and Harchol-Balter 20061. 


ISchrage 19681. 
tile 


SRPT 


to implement 


based 

SRPT. 


I Biersack et al. 2007)1 and IIAlSa’deh and Adnan 20081 have been proposed for the 
scheduling of HTTP requests arriving at web server. Still these algorithms are not 
used in practice. There are several reasons for this. One reason is that SRPT causes 
starvation for large size requests (because of its tendency to favor small size requests) 
in the presence of continuous arrival of small sized requests. Also, SRPT relies 
completely on size of requested file to determine the mean response time which is 
not enough because it does not take into consideration the user server interaction 
parameters present in Internet like throughput of the user connection. Further, SRPT 
based policies cannot be applied for dynamic HTTP requests where file size is not 
known in advance. 


5.2. Related Work 

L. Cherkasova introduced the concept of Alph a-Scheduling to improve the response 
time for web applications ICherkasova. 199^ . This strategy lies between FIFO 
(first-in-first-out) and SRPT. The measure of balance between FIFO and SRPT was 
decided by the parameter alpha.The major drawback of this strategy is that they have 
not taken into consideration the user-server interaction parameters over Internet like 
network bandwidth, which may play a crucial role in influencing the response time of 
the user. 

It is well known from queuing theory that SRPT is an optimal algo rithm for mini - 
mizing the mean response time when the job size is known in advance | Schrage 19681. 
For static requests, the size of the request (time required to service the request) can 
be well-approximated by the size of the file requested by HTTP request, which is 
known to the server. The SRPT based scheduling strategies uses this approximation 
for the scheduling of HTTP requests. The work done to implement SRPT scheduling 
for web servers has been done o n both the application level and the kernel level. 
M. Crovella and R. Frangioso | [Crovella and Frangioso 1999| | implemented SRPT 
connection based scheduling (priority to requests with smaller size) at the application 
level and got improvement in the response time as compared to traditional PS 
scheduling but a t the cost of drop of throug hput by some constant factor. Later, M. 
Harchol Balter IHarchol-Balter et al. 20031 implemented SRPT based scheduling 
strategy at the kernel level a nd got better improvements in response time than in 
I Crove^lla and Frangioso 1999| and the th roughput problem was eliminated. 

C. Murta IIMurta and Corlassoli 20031 introduced an extension to SRPT known as 
FCF (Fastest Connection First) scheduling which takes into consideration the network 
conditions instead of relying only on file size of request. The strategy gives priority 
to the requests with shorter size issued through faster connections.The information 
sharing between a Web server and the TCP connections running in the server was 
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taken into consideration. They got improvements in the response time as compared 
to SRPT, although SRPT provided the shortest server delay. They concluded that 
scheduling strategies for weh servers should take into consideration the network 
conditions (WAN) and use t hem to set up priorities for the requests. Ahmad AlSa’deh 
lAlSa’deh and Adnan 20081 suggested SRRT (Shortest Remaining Response Time) 
scheduling for weh servers which is also an extension to SRPT. In addition to the file 
size, the scheduling algorithm also takes into consideration current RTT (Round-Trip 
Time) and TCP congestion window size for the servicing of HTTP requests. They 
got an aver age improvement of about 7.5% over SRPT algorithm. For the evaluation 
of results, HAlSa’deh and Adnan 20081 did not take into account the variability of 
network bandwidth from user to user but instead used same network bandwidth for 
all the clients throughout the experiments. Further SRRT scheduling strategy was 
applicable only for static requests. 

Most of the papers have considere d th e idea of SRPT based scheduling 
[Crovella and Frangioso 1999||, 

Corlassoli 20031, IlCherk 


^_ IlSchroeder and Harchol-Balter 2006L 

IlCherkasova. 19^ . 


IBiersack et al. 20071 . 


i )olicies _ 

Murta and (!)orlassoli 200317 

lAlSa’deh and Adnan 20081 . But these scheduling strategies do not prevent the 
starvation of large size requests in the presence of continuous arrival of short size 
requests. Further, all SRPT based policies are applicable only for the static HTTP 
requests where the file size is known in advance. But today in most of the practical 
scenarios, web servers use dynamic content as well (cgi and other non static requests). 
So, SRPT based policies cannot be applied in t hese scenarios. 

Bender, Chakravati and Muthukrishnan [Bender et al. 199^ proved that SRPT 
will cause large files to have arbitrarily high response time. This paper raised an 
important point that while choosing a scheduling policy it is important to consider 
not only the scheduling policy’s performance but also whether the policy is fair, i.e. 
some of the jobs have arbitrarily high response time. There have been some research 
papers which studi es the types of unfairness ca used by SRPT. For example, M. Gong 
and C. Williamson |Gong and Williamson 2003| investigates about two different types 
of unfairness (endogenous and exogenous) caused by Shortest Remaining Processing 
Time (SRPT) strategy. 

In order to overcome the problems related to SRPT based policies, in this work 
we present simple but effective scheduling policies with dynamic priority adjustment 
which addresses the problem of starvation and keeping the mean response time low, 
both simultaneously. Further, the policy will be able to handle the dynamic HTTP 
requests where the file size is not known in advance. 

5.3. Speed Up Scheduling 

We present two scheduling algorithms named as SSU (Static Speed Up) and DSU 
(Dynamic Speed Up) for static (file size is known in advance) and dynamic environ¬ 
ments (file size is not known in advance) respectively with a slight modification in the 
priority function defined in eql4]based on implicit speed up. The algorithms are simple 
that assigns priority to the requests based on their specific characteristics. The request 
with lowest value of priority function is accelerated to the server and thus, is chosen 
for next service. 

5.3.1. Static Speed Up (SSU) scheduling:. This algorithm is non preemptive. Let AT{r), 
FS{r) and LB{r) respectively denote the arrival time of request r, file size requested 
by r and link bandwidth of the connection of the user sending request r to the server 
i.e. throughput of the user connection. The priority for request r is assigned as follows: 


Priority (r) = 


AT{r) X FS{r) 
LB{r) 


(16) 
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The request with minimum value of priority function is chosen next for service. If 
there are multiple such requests, then the one with shortest file size is chosen. The 
idea is to give priority to small size requests issued through faster connection in order 
to improve the mean response time hut at the same time taking into consideration 
the arrival time of request in order to address the problem of starvation. The SSU 
algorithm is applicable only for static requests arriving at web server since it includes 
the file size of request in its priority function. 

5.3.2. Dynamic Speed Up (DSU) scheduling:. In order to schedule dynamic requests ar¬ 
riving at web server, we make use of LAS (Least Attained Service) policy since the 
request size is not known in prior. LAS is a preemptive scheduling policy which tries 
to predict the remaining job size by the amount of service, job has received so far and 
thus, mimics SRPT by giving preference to least attained service job. LAS will create 
starvation for large size requests since arriving jobs will always have low value of at¬ 
tained service and thus, jobs existing in queue for long period of time will not get time 
to be serviced. Thus, we also take into consideration the arrival time of jobs to address 
this problem of starvation. 

Let AT{r), AS{r) and LB{r) respectively denote the arrival time of request r, the 
attained service for r (amount of time it has received service so far) and link bandwidth 
of the user connection sending request r to the server. Then the priority for request r 
is assigned as follows: 

AT(r)xAS(r) 

Priority {r) = - - (17) 

LB{r) 

The DSU scheduling algorithm is preemptive. We discuss the impact of the three 
parameters that we keep in the priority function on web requests : 


— Impact of Arrival Time in Priority: The term AT{r) in eg. 1161 and 1171 ensures 
that every request will eventually be served. This is because AT (r) keeps on 
increasing for the incoming requests with respect to time and we choose the request 
with minimum value of priority to get next service. Thus, the time will always 
come when the request waiting for too long will have the minimum value of priority 
function. Hence, the starvation for any request is impossible. 

Consider a request ri with a large FS{ri), arriving at time AT{ri). Since 
requests keep coming, and arrival time of requests keep increasing, there exists a 
request r 2 , such that implying that ri will be served 

before r 2 . That is, if AT{r 2 ) > AT{ri) and served 

before r 2 . Since remaining file sizes are finite and decreasing, and link bandwidth 
is also constant, each request will definitely be served as priority of new requests 
keep increasing as their arrival time is increasing. 

— Impact of file size in Priority: If there are various requests having near about 
similar arrival time issued through the same link bandwidth connection, then the 
request with least file size is preferred over others (like SRPT) in order to improve 
the response time. 

Consider a small size request ri with very small value of FS{rl) arrived at 
time AT{ri). Let there be another request r 2 already present in the request 
queue such that AT{r 2 ) < AT{ri) and FS{r 2 ) is much larger than FSiri). If 
^ , then Ti will be served before r 2 . It indicates that 

smaller value of FS{ri ) (resulting in lower priority function value for ri than r 2 
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) results in preferring ri over r 2 for next service. Each request r will get served 
because there will be certain arrival time AT{ri) (for some request r/ ) such that all 
the requests after that irrespective of their file size (> 1) will have higher value of 
priority function than r. 

— Impact of Link Bandwidth in Priority: Link Bandwidth plays a very crucial role 
in influencing the response time for web clients. Greater the link bandwidth, lesser 
is the response time. If there are set of requests present in the request queue having 
similar arrival time and near about same file size, then the one with the fastest link 
bandwidth is preferred over others which will result in lesser response time. 

Consider a request ri with arrival time AT{ri) issued through fast connection 
of say 100 Mbps. Then another request r 2 arrives with smaller value of FS{r 2 ) as 
compared to FS{ri) issued through a relatively slower connection of say 1 Mbps such 
that ^ (because LB{ri) is much greater than LB[r2) ), 

then ri (fast connection request) will be served before r 2 (slow connection request) 
showing that how link bandwidth can impact the value of priority function. Each 
request r (even low bandwidth request) will get served irrespective of other requests 
because there exist an arrival time beyond which their value of priority function is 
higher than request r assuming some maximum link bandwidth (which is finite). 

5.4. Experiments and Results 

We used a simple simulation model using trace-driven workload for the evaluation 
of results. We used a real one day logs of web requests arriving at International Insti¬ 
tute Of Information Technology, Hyderabad proxy server collected on 9th September, 
2012. Our workload consisted of a part of single day trace consisting of about 1 million 
requests. Each entry in the trace described a request made to the server and contained 
information about the request like arrival time of the request, URL requested, size of 
the request and other status information. The file size in our workload ranged from 
56 bytes to 15 MB. 

We used a Scheduler Simulator in order to simulate all the requests arriving at 
a web server present in the trace based workload. Each request present in the trace 
can be considered as a process whose burst time is directly proportional to the file 
size requested and is inversely proportional to the link bandwidth of the request 
connection. The RAM of the system is 4GB and processor speed is 3.3GHz. The 
requests were simulated as per their information in the trace based workload using 
the Scheduler Simulator. 

For the purpose of assigning network bandwidth to each of the request, we parti¬ 
tioned the requests into following three types of classes : 


— Small files (0 - 50 KB): These smaller sized requests contributed towards 64% of the 
total workload. 

—Medium-sized files(50 KB - 500 KB): These requests contributed towards 32% of the 
total requests present in our workload. 

—Large files (> 500 KB): These requests contributed towards 4% of the overall work¬ 
load. 

We performed our experiments for two different types of extreme scenarios. 


— In Scenario 1, we assigned 1 Mbps, 10 Mbps and 100 Mbps of network bandwidth 
to Small, Medium-sized and Large files respectively. 
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Table XVII. Overall mean and maximum response time (in sec) for both the 
scenarios for different scheduling strategies. 


Algorithm 

Scenario 1 

Scenario2 

Mean 

Max 

Mean 

Max 

PS 

2.37 

4.44 

1.03 

6.16 

SRPT 

1.58 

4.39 

0.5 

6.18 

“SSU 

T:33 

4.18 

0.6 

“5:gs 

^SU 

2.12 

4.26 

0.98 

6.08 


Table XVIII. Scenario 1 lOverall mean and maximum response time (in sec) for different types of requests for different 
scheduling strategies. 


Algorithm 

sisin 

Medium 

Large 

Mean 

Max 

Mean 

Max 

Mean 

Max 

PS 

2.26 

4.41 

2.62 

4.44 

1.38 

4.39 

SRPT 

0.64 

2.71 

3.13 

4.22 

4.3 

4.39 

“SSU 

1.23 

4.10 

1.55 

4.18 

0.77 

3.03 

^Su 

2.04 

4.19 

2.59 

4.26 

1.67 

4.13 


— In Scenario 2, we assigned 1 Mbps, 10 Mbps and 100 Mbps of network bandwidth 

to Large, Medium-sized and Small files respectively. 

5.4.1. Scenario 1:Results 

— Small files (0-50 KB) assigned to 1 Mbps connection. 

— Medium-sized (50 KB-500 KB)files assigned to 10 Mbps connection. 

—Large (> 500 KB) files assigned to 100 Mbps connection. 

Table lXVlIl shows that both the SSU and DSU scheduling algorithms outperforms 
the existing scheduling algorithms in terms of maximum response time validating our 
idea that these algorithms provide less scope for starvation. The reason behind this 
decrease, is that our scheduling algorithms take into consideration the arrival time 
(aging) of request which do not allow any request to stay for a very long period of 
time. SSU scheduling provides the minimum response time in Scenario 1 even bet¬ 
ter than SRPT scheduling. The reason behind this improvement is that SRPT prefers 
the shorter size requests independent of their slow network bandwidth (1 Mbps) con¬ 
nection in this scenario. However, the actual response time suffered by the client is 
influenced by both the size of the requested file and the link bandwidth. Greater the 
link bandwidth, lower is the response time. SSU scheduling takes into consideration 
both the size of requested file as well as the link bandwidth of the user connection. 
Note that, such improvements are not observed in DSU algorithm, because it does not 
know in prior the file size of request. It uses attained service measure for the estima¬ 
tion of file size. However, it outperforms PS scheduling (which also do not require file 
size) in mean as well as max response time. The major advantage of DSU scheduling 
is that it can be used for the scheduling of dynamic web requests (file size of request is 
not know n in pri or) whereas SRPT and SSU are not applicable in such cases. 

Table IXVIIII shows the mean and maximum response time for the different types 
of requests for different scheduling strategies for Scenario 1. The small size requests 
get a very low mean response time for SRPT (because of it’s tendency to favor small re¬ 
quests), whereas on the other hand, large size requests (connected through 100 Mbps 
link bandwidth) suffer from very high mean response time of 4.3 seconds by SRPT. 
SRPT only takes into consideration the remaining size of requested file thereby mak¬ 
ing the large size jobs to wait for too long. Such high penalization for large requests is 
not observed for both SSU and DSU scheduling because along with the file size, they 
also consider the arrival time in priority to avoid starvation. 

From the above results, it can be concluded that in Scenario 1, SSU strategy out- 
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Table XIX. Scenario 2:Overall mean and maximum response time (in sec) for different types of requests for different 
scheduling strategies. 


Algorithm 

Small 

Medium 

Large 

Mean 

Max 

Mean 

Max 

Mean 

Max 

PS 

0.12 

0.61 

2.18 

3.42 

5.65 

6.16 

SRPT 

0.006 

0.09 

1.05 

2.9 

4.48 

6.18 

“5SU 

0.07 

0.2 

1.17 

2.9 

4.4 

5.98 

^5U 

0.27 

0.68 

2.12 

3.88 

5.4 

6.08 


performs SRPT as well as PS scheduling in terms of mean response time without cre¬ 
ating starvation like SRPT. DSU scheduling also provides less scope of starvation and 
outperforms PS scheduling with a major advantage of its applicability in serving of 
dynamic web requests. 

5.4.2. Scenario 2:Results 

— Small files (0-50KB) assigned to 100Mbps connection. 

— Medium-sized (50-500KB) assigned to 10Mbps connection. 

—Large files (> 500KB) assigned to 1Mbps connection. 

The starvation caused b y SRPT fo r larg e files (connected through 1 Mbps band¬ 
width) is visible from the Table IXVTIl and IXIXI for Scenario 2. SSU and DSU scheduling 
performs better than SRPT scheduling in terms of maximum response time indicating 
the reduction in starvation. However, in this scenario the overall mean response time 
is minimum for SRPT better than SSU as well as DSU scheduling. This is because 
in this scenario, smaller size requests are connected through faster link bandwidth 
connection. Thus, by preferring only the smaller size requests, SRPT is also indirectly 
giving preference to faster connection requests (because smaller size requests are con¬ 
nected through faster connections). SSU scheduling gives preference to both shorter 
size requests as well as earlier arrived requests. Thus, it’s response time is found to be 
slightly lower than SRPT. In this case, SRPT provides an optimal response time. We 
can say that this scenario is the best case scenario for SRPT scheduling. 

5.5. Discussion 

We have considered two extreme case scenarios for the variability in network band¬ 
width for different users. In first scenario, smaller size requests and larger size re¬ 
quests are respectively connected through slower and faster link bandwidth connec¬ 
tion while in the second scenario vice-versa. 

In Scenario 1, SSU scheduling outperforms both PS and SRPT scheduling. SRPT 
scheduling blindly gives preference to shorter size requests irrespective of taking into 
account their slower network bandwidth in this scenario. It leads to the starvation 
of large size requests which are connected through faster link bandwidth connec¬ 
tion. SSU scheduling takes into consideration both the size as well as link bandwidth 
thereby, improving the mean response time from 1.53 to 1.3 sec. SSU scheduling re¬ 
duces the problem of starvation in terms of maximum response time as compared to 
SRPT scheduling from 4.39 to 4.18 sec because it takes into consideration the arrival 
time of request and thus, prevents the request from waiting for too long. DSU schedul¬ 
ing outperforms PS scheduling in terms of mean response time as well as max response 
time. DSU scheduling can further be applied for dynamic environments whereas SSU 
and SRPT cannot. 

In Scenario 2, small size requests are connected through faster connection while 
the large size requests are connected through slower connection. SRPT scheduling 
while giving preference to small size requests is also indirectly giving preference to the 
requests with faster connection (because of the nature of this scenario). Thus, SRPT 
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Table XX. Scenario 1: Percentage of speeded up and slowed 
down requests for different scheduling strategies. 


Strategy 

% of achieved speed-up 

% of slow down 

PS 

49.8% 

50.2% 

SRPT 

61.2% 

37.6% 

SSU 

62.6% 

37.2% 

DSU 

51.2% 

48% 


Table XXI. Scenario 2: Percentage of speeded up and 
slowed down requests for different scheduling strategies. 


Strategy 

% of achieved speed-up 

% of slow down 

PS 

85.25% 

14.75% 

SRPT 

91.75% 

8% 

SSU 

91% 

7.5% 

DSU 

86.25% 

13.75% 


Table XXII. Percentage Of User Abort in two scenarios for dif¬ 
ferent scheduling strategies. 


Strategy 

Scenario 1 

Scenario 2 

PS 


2T% 

SRPT 

3% 

1.8% 

SSU 


0.25% 

DSU 

1.6% 

1.1% 


provides an optimal response time in this scenario (best-case scenario for SRPT). SSU 
scheduling suffers a drop from 0.5 sec (as in SRPT) to 0.6 sec in terms of mean re¬ 
sponse time as compared to SRPT scheduling. However, SSU scheduling again reduces 
the problem of starvation from 6.18 to 5.98 sec in terms of maximum response time as 
compared to SRPT. DSU scheduling outperforms PS scheduling in terms of mean as 
well as maximum response time in this scenario as well. The reason behind this im¬ 
provement is that PS scheduling do not give any preference to short size requests how¬ 
ever, DSU tries to mimic SRPT by giving preference to least attained serviced request. 

Table IXXI and IXXII indicates the percentage of speeded up and slowed down 
requests iv.r.t. FCFS scheduler for Scenario 1 and 2 respectively. SSU and SRPT 
scheduling dominates in terms of % of achieved speed up as compared to other 
scheduling algorithms for both the scenarios. DSU performs better than PS schedul¬ 
ing in both the scenarios. 


5.6. User Abort Analysis 

For starvation analysis, we used Maximum Response Time metric as a measure of 
unfairness till now. Now, we are going to analyze starvation for the different schedul¬ 
ing strategies using another metric, i.e. User Abort. 

Today, users cannot wait for large amount of time for getting their request serviced. 
They might look for other alternatives if some particular website makes them wait for 
too long. Thus, we may assume that the user waits for certain specific amount of time 
(say user threshold) till his request gets served after which he aborts. This gives rise 
to a situation known as User Abort. 

We set the value of user threshold to be slightly less than the maximum response 
time for unbiased PS scheduling and measure the percentage of users having response 
time greater than user threshold (percentage of user abort) for the PS, SRPT, SSU and 
DSU sche duling strategies. 

Table IXXlIl clearly indicates that SRPT penalizes the large size requests whereas 
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on the other hand, SSU and DSU, both strategies provide less scope of starvation 
thereby handling more number of users. SSU and DSU scheduling outperforms both 
PS and SRPT in terms of user aborts and thus, these algorithms ensures more reliable 
service than either PS or SRPT. 

6. APPLICATIONS: CPU SCHEDULING 

We extend our priority functions designed for implicit speed up for the purpose of 
CPU Scheduling and demonstrate the effectiveness of our policies using simulation 
based model. 


6.1. Introduction 

In real time uniprocessor multiprogrammed systems, only one process can run at a 
time. All the processes which are waiting for CPU resources and are ready to execute 
resides in the ready queue. Whenever CPU becomes idle (either because executing pro¬ 
cess is waiting for I/O routine to complete or process terminates), a new process needs 
to be selected for execution among all the processes present in the ready queue. The 
task of selecting a new process among all the runnable processes present in the ready 
queue for execution is referred to as CPU Scheduling. The module that is responsible 
for giving control of the CPU resources to the selected process by CPU Scheduler is 
referred to as dispatcher. CPU Scheduling is a fundamental problem in terms of min¬ 
imizing the mean wait time, ensuring fairness among processes and enhancing the 
overall performance of the system. 


6.2. Overview of existing CPU Scheduiing Aigorithms 

Many CPU scheduling algorithms such as FCFS, Shortest Job First, Round-Robin, 
Priority Scheduling etc. have already been proposed. First Come First Serve (FCFS) 
scheduling selects a job with least arrival time. This algorithm is fair to all and will 
not create starvation for any of the job. However in FCFS, the earlier arrived process 
with large duration time may unnecessarily delay the short jobs arrived later leading 
to high mean wait time. This phenomena is referred to as Convoy Effect. Shortest Job 
First algorithm selects a process with least duration time among all the runnable pro¬ 
cesses. SJF al gorithm is prov ably optimal in giving minimal average wait time for all 
the processes I Schrage 1968| |. But the tendency of SJF algorithm to prefer shorter jobs 
might create starvation for large jobs. Thus, SJF algorithm might not be fair towards 
long processes in the presence of continuous arrival of short processes. Further, prac¬ 
tically process duration time is not known in advance and thus, SJF algorithm cannot 
be applied directly for the purpose of CPU scheduling. There are heuristics like expo¬ 
nential average which predicts the next CPU burst time of a process depending upon 
its previous CPU burst time. Round Robin scheduling is similar to FCFS scheduling 
but preemption is added to allow switching between processes. In RR algorithm, each 
of the process is given a fixed time slice known as time quantum. The process runs 
until the time quantum expires, then context switch takes place leading to the execu¬ 
tion of the next arrived process and so on. Very large value of time quantum will make 
Round Robin exactly similar to FCFS algorithm. If the value of time quantum is too 
less, then most of the time would be spent in context switching between the processes. 
The average wait time under RR policy is often long. Priority based scheduling selects 
a process with highest priority to schedule next. This scheduling technique may cre¬ 
ate starvation for low priority processes if high priority processes keep coming with 
respect to time. 

Wait time and Starvation (situation of waiting for indefinite time) are two of the 
most important factors for choosing a CPU Scheduling policy for processes. We present 
two Speed Up scheduling techniques for the purpose of CPU Scheduling which aims 
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Table XXIII. Mean Walt Time (in time units) for different CPU Scheduling policies based on different system load values. 


System load (p) 

FCFS 

SJF 

5RPT 

RR 

SSUPS 

DSUPS 

0.8 

74 

37 

27 

78 

38 

77 

0.9 

129 

57 

45 

129 

55 

126 

1 

135 

60 

49 

143 

63 

132 

1.1 

672 

188 

170 

628 

194 

600 

1.2 

532 

255 

238 

868 

273 

850 

1.3 

1543 

443 

436 

1451 

520 

1497 

FED 

3425 

gsT 

551 

28T9 

n22 

25T1 


Table XXIV. Maximum Wait Time (in time units) for different CPU Scheduling policies based on different system load values. 


System load (p) 

FCFS 

SJF 

SRPT 

RR 

SSUPS 

DSUPS 

0.8 

455 

1448 

1461 

1059 

1420 

1802 

0.9 

589 

1554 

1713 

1417 

1520 

2397 

1 

695 

1823 

1823 

1548 

1802 

2051 

1.1 

1796 

7887 

7894 

6759 

6453 

8380 

1.2 

2351 

8979 

8979 

7170 

6658 

8395 

1.3 

3371 

10671 

10671 

9233 

7262 

8774 

PLD 

7053 

15431 

16529 

15495 

13051 

14186 


at reducing mean wait time but at the same time ensuring less scope of starvation - 
Static Speed Up Process Scheduling (SSUPS) (assumes process burst time is known) 
and Dynamic Speed Up Process Scheduling (DSUPS) (process burst time is not known 
in advance). 

6.3. Speed Up Process Scheduling Algorithms 

6.3.1. Static Speed Up Process Scheduling (SSUPS). This scheduling technique is non- 
preemptive. The priority for a process p is assigned as follows: 

Priority{p) = AT(p) * BT(p) (18) 

where AT{p) denotes the arrival time of process p and BT{p) denotes the CPU burst 
time of process p. The process with least value of priority is chosen to schedule next. 
If there are multiple processes with minimum priority value, then the process with 
smallest burst time is chosen. This scheduling technique assumes that process CPU 
burst time is known in advance. The idea is to give preference to small processes but 
this preference takes into consideration the arrival time to avoid starvation. 

6.3.2. Dynamic Speed Up Process Scheduling (DSUPS). DSUPS scheduling technique is 
preemptive and is more practical than SSUPS because it does not make use of CPU 
burst time to assign priority but instead use attained service parameter to estimate 
the CPU burst time of the process. The priority for process p is assigned as follows: 

Priority (p) = AT{p) * AS{p) (19) 

where AT(p) denotes the arrival time of process p and AS(p) denotes the attained 
service duration for process p. The process with minimum priority value is chosen. 

6.4. Experiments and Results 

We used a simulation based model for conducting experiments and getting results. 
We assumed that processes are arriving in Poisson distribution and their burst times 
are exponentially distributed. The number of processes for experiments were 10 thou¬ 
sand. We conducted experiments for different values of process load p and present here 
the results for the same. We also used real process logs dataset (PLD) used in Chapter 
5 for comparison between different scheduling algorithms. 

Experimental results show that SSUPS scheduling algorithm provides far better 
mean wait time as compared to FCFS and Round Robin scheduling but slightly higher 


ACM Journal Name, Vol. V, No. N, Article A, Publication date: January YYYY. 

































A:29 


Table XXV. Standard Deviation of wait time for different CPU Scheduling policies based on different system load values. 


System load (p) 

FCFS 

SJF 

SEPT 

EE 

SSUPS 

DSUPS 

0.8 

94 

107 

110 

144 

105 

224 

0.9 

19(3 

152 

171 

250 

ijg 

351 

1 

197 

168 

180 

268 

165 

349 

1.1 

879 

781 

832 

1125 

714 

1474 

1.2 

I2D8 

roiD 

roi5 

1474 

52D 

1575 

1.3 

1835 

1526 

1536 

2080 

1311 

2154 

FED 

22B5 

2826 

2782 

3293 

2253 

3610 


than SJF and SRPT (which are optimal algorithms for providing minimal mean wait 
time). Further, our SSUPS algorithm reduces the problem of starvation in terms of 
maximum wait time as observed in SJF and SRPT (preemptive version of SJF) for all 
values of system load since our algorithm takes into consideration the arrival time 
to avoid large delay for any of the process. On the other hand, SJF and SRPT makes 
large size process to wait for too long resulting in high value of maximum wait time. 
Low value of maximum wait time guarantees that all the processes get good service. 
DSUPS algorithm is also successful in providing better mean wait time as compared 
to FCFS and Round Robin for majority of system load values and provides lesser max¬ 
imum wait time as compared to SJF and SRPT under overload situation (p >= 1.2). 
FCFS algorithm provides the least maximum wait time since it is fair to all the pro¬ 
cesses but at the expens e o f high mean wait time. By examining the experimental 
results from Table DOCIIII and IXXIVl we can conclude that our process scheduling algo¬ 
rithms (SSUPS and DSUPS) are successful in providing less mean wait time but at the 
same time ensuring less scope of starvation. Thus, our proposed algorithms addresses 
the trade-off between wait time and starvation. SSUPS algorithm performs much bet¬ 
ter than DSUPS algorithm in terms of both mean as well as maximum wait time. The 
reason is that SSUPS algorithm assumes that process burst time is known in advance 
whereas DSUPS algorithm tries to estimate the process burst time using attained ser¬ 
vice parameter. But DSUPS algorithm is more practical than SSUPS algorithm since 
process b urst time is not known in adv ance. 

From ISilberschatz and Peter 200911 . regarding interactive systems (such as time 
sharing systems), it is stated that minimizing the variance measure for wait time is 
more important than to minimize the mean measure. A system with reasonable and 
predictable waiting time may be consider ed mo re desirable than a system that is faster 
on average but is highly variable. Table IXXVI shows the value of standard deviation 
(square of standard deviation is variance) in wait time for different CPU scheduling al¬ 
gorithms based on different values of system load. We experimentally find that SSUPS 
algorithm provides the minimal variance in wait time and thus, outperforms all the ex¬ 
isting algorithms in terms of variance minimization. 

6.5. Speed-up/Slow-down Analysis 

Speed Up concept provides a new evaluation criteria for comparing different 
scheduling algorithms based on their speed-up/slow-down characteristics. We analyze 
the existing scheduling algorithms from the perspective of speed up in this sub-section. 

Consider a set of processes < pi, P2, Ps , . , Pn >■ Let their corresponding wait 

times under scheduling policy X and Y be denoted by < , Wx^ , ., Wx^ > and 

< Wyj, Wyj, Wy^ ,., Wy^ > respectively. 

Definition 6.1. A scheduling algorithm X is said to speed up process pi w.r.t schedul¬ 
ing algorithm Y iff Wxi — Wy^ < 0. Similarly, X is said to slow down process pi w.r.t. Y 

iff Wxi — Wy^ > 0. 
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Table XXVI. Percentage of achieved speed up (SU) and slo\w down (SD) for different CPU scheduling strategies with 
respect to other scheduling strategies. 


Algorithm 

FCFS 

RR 

SJF 







FCFS 

0% 

0% 

34.8% 

61.4% 

9.4% 

82.8% 

RR 

61.4% 

34.8% 

0% 

0% 

8.8% 

87% 

ydb' 

82.8% 

9.4% 

~WWc 


~Wc 

~Wo 

SRPT 

88% 

8% 

93.8% 

1.8% 

75.4% 

14.4% 

“ssn 

81.6% 

10.6% 

87.2% 

8.2% 

16.6% 

42.2% 

^su 

72.8% 

24.4% 

76.8% 

19.6% 

48% 

47.8% 


Table XXVII. Percentage of achieved speed up (SU) and slow down (SD) for different CPU scheduling strategies with 
respect to other scheduling strategies. 


Algo. 

SRPT 

5SU 

DSU 





SU 


FCFS 

8% 

88% 

10.6% 

81.6% 

24.4% 

72.8% 

RR 

1.8% 

93.8% 

8.2% 

87.2% 

19.6% 

76.2% 

“SJF 

14.4% 

75.4% 

42.2% 

16.6% 

47.8% 

48% 

SRPT 

0% 

0% 

78.2% 

13.8% 

60.8% 

8.8% 

“SSU 

13.8% 

78.2% 

~Wo 

“0% 

47.4% 

48.6% 

^Su 

8.8% 

60.8% 

48.6% 

47.4% 

0% 

0% 


Table llSVIl and lXXVIIl presents the comparison analysis for different CPU Schedul¬ 
ing algorithms based on their speed-up/slow-down characteristics for p = 1.1. Round 
Robin speeds up 61.4% of the processes w.r.t. FCFS scheduling. All the four schedul¬ 
ing algorithms SJF, SRPT, SSUPS and DSUPS speeds up nearly 70-90% of the pro¬ 
cesses w.r.t. FCFS and Round Robin Scheduling. SRPT scheduling speeds up ma¬ 
jority of the processes w.r.t. any other scheduling strategy. SSUPS scheduling w.r.t. 
DSUPS scheduling speeds up 47.8% processes and slows down 48.6% processes. In 
this manner, we can compare between any two scheduling algorithms based on their 
speedup/slowdown characteristics. 


6.6. Discussion 

Speed Up concept provides a new mechanism to compare and contrast differ- 
ent scheduling algo rith ms. Speed Up can be implemen ted by various techniques. In 
IIKafeza et al. 2006]| and [ Kafeza and Karlapalem 2000b |, it was implemented by loca¬ 
tion table and swapping of jobs. In this paper, we use priority function to seamlessly 
and certainly perform speed up. Any scheduling algorithm that uses ranking function 
to select next job can potentially speed up that job. 

From speed up perspective, we have assumed that no process is requesting for speed 
up. We extended our implicit priority functions of speed up problem for the purpose of 
CPU Scheduling. Experimental results show that our proposed techniques are suc¬ 
cessful in reducing the mean wait time and at the same time providing less scope of 
starvation, thus addressing the trade-off between wait time and starvation. Further, 
SSUPS algorithm provides minimal variance in wait time as compared to all the other 
existing scheduling algorithms. 


7. CONCLUSION AND FUTURE WORK 

Today in flexible and dynamic application environments, user might request for 
faster execution of some already executing instances. In such cases, the system should 
be able to respond to such on-line requests for speed up. The problem of scheduling of 
speed up requests at run-time has not been adequately studied in literature before. 
In this paper, we modeled the Speed Up Scheduling problem without acquiring 
additional resources to handle on-line speed up requests and analyzed two different 
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aspects of it - Speed Up with no constraints and Fine Grained Selective Speed Up. 
We provided efficient implicit techniques to address speed up problem where the 
notion of speed up is incorporated in the priority function. Experimental results show 
that our proposed algorithms are able to provide almost the similar achieved speed 
up as compared to existing speed up algorithms. Further, our speed up algorithms 
outperforms the existing algorithms in terms of slow down caused to non-urgent jobs 
thereby providing remedies for the delayed jobs alike existing algorithms where speed 
up is achieved for urgent jobs at the expense of high slow down for the rest of the 
jobs. Overall mean wait time is improved dramatically by our speed up algorithms 
owing to the design of the priority function. We provided GPSU algorithm to address 
more specific case of speed up problem - Fine Grained Selective Speed Up (each of the 
urgent job could request for specific percentage of speed up at run-time) which has not 
been yet addressed. Our algorithms are computationally efficient than existing speed 
up algorithms where there is an overhead of maintaining a location table. 

For web scheduling, the performance of SRPT degrades dramatically in an en¬ 
vironment where there is a high variability in link bandwidth. Using our implicit 
techniques for speed up problem, we provided two web scheduling algorithms SSU and 
DSU for static and dynamic environments respectively. We established the usefulness 
of our algorithms with a simulation based model using trace driven workload. We 
extended our algorithms for CPU Scheduling and experimentally finds that our 
algorithms are successful in reducing the mean as well as variance in wait time and 
at the same time providing less scope for starvation. 

In this work, we provided a framework for addressing speed up related problems. 
But still there are several open issues that need to be addressed. Throughout this 
work, we assumed that no additional resources could be acquired. An extension of our 
work could be to examine how such on-line requests for speed up can be handled if 
multiple servers and other additional resources are available so as to maximize the 
number of speeded up urgent jobs but at the same time ensuring efficient resource 
utilization. We also assumed that jobs are independent of dependence constraints 
throughout this work. It would be interesting to see how we can speed up jobs 
requesting for it where that job is either dependent on some other non-urgent job or 
on another job requesting for speed up ? In the presence of multiple queues, it would 
be interesting to see how load balancing could be used in achieving speed up. 
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